Hand tracking refers to the process of detecting the poses of a user's hand. For example, cameras can be set up to capture images of the user's hand. The images may be processed to determine a hand pose, which may be used as an input to for a computing device. The speed and accuracy of the programmatic hand pose analysis from images depends on various factors, such as the ability to recognize variations in hand size and shape across different users. It is desirable to provide highly accurate hand model calibration and hand pose identification from images, while reducing computational time and hardware requirements.